Perceptual Resolution of Ambiguity: Can Tuned, Divisive Normalization Account for both Interocular Similarity Grouping and Difference Enhancement.

Our visual system usually provides a unique and functional representation of the external world. At times, however, the visual system has more than one compelling interpretation of the same retinal stimulus; in this case, neural populations compete for perceptual dominance to resolve ambiguity. Spatial and temporal context can guide perceptual experience. Recent evidence shows that ambiguous retinal stimuli are sometimes resolved by enhancing either similarity or differences among multiple percepts. Divisive normalization is a canonical neural computation that enables context-dependent sensory processing by attenuating a neuron's response by other neurons. Experiments here show that divisive normalization can account for perceptual representations of either similarity enhancement (so-called grouping) or difference enhancement, offering a unified framework for opposite perceptual outcomes.

Ambiguity is a linking feature for interocular grouping.

Ambiguity is implicit in neural representations of the physical world. Previous work has examined how the visual system resolves ambiguous neural signals that represent various features, such as the percept resulting from rivalrous chromaticities or forms. Relatively little is known, however, about the contribution of unambiguous neural representations to perceptual resolution of ambiguous ones. This is addressed here by measuring perceptual resolution of ambiguity by grouping, which is operationalized as the tendency for multiple similar ambiguous representations to be seen as identical to each other. Multiple chromatically ambiguous representations were created using interocular switch rivalry and presented together with a nearby but separate unambiguous (non-rivalrous) chromaticity. The magnitude of grouping the chromatic regions was compared when ambiguous regions were seen alone versus with unambiguous regions seen simultaneously. Contrary to prevailing theory that the resolution of the ambiguous percepts would follow the unambiguous ones, the ambiguous chromatic regions consistently appeared identical to each other, but their appearance was not found to be attracted to the unambiguous color percept. This supports the proposition that the ambiguity itself in a neural representation is a linking feature contributing to perceptual disambiguation.

The Certainty of Ambiguity in Visual Neural Representations.

Some images evoke bistable percepts: two different visual experiences seen in alternation while continuously viewing an unchanged stimulus. The Necker Cube and Rubin's Vase are classic examples, each of which gives alternating percepts of different shapes. Other bistable percepts are alternating colors or directions of motion. Although stimuli that result in salient bistability are rare and sometimes cleverly constructed to emphasize ambiguity, they have been influential for over 150 years, since the work of von Helmholtz, who considered them to be evidence for perceptual visual processes that interpret retinal stimuli. While bistability in natural viewing is uncommon, the main point of this review is that implicit ambiguity in visual neural representations is pervasive. Resolving ambiguity, therefore, is a fundamental and ubiquitous process of vision that routinely affects what we see, not an oddity arising from cleverly crafted images. This review focuses on the causes of widespread ambiguity, historical perspectives on it, and modern knowledge and theory about resolving it.

Binocularly-driven competing neural responses and the perceptual resolution of color.

Competing rivalrous neural representations can be resolved at several levels of the visual system. Sustained percepts during interocular-switch rivalry (ISR), in which rivalrous left- and right-eye stimuli swap between eyes several times a second, often are attributed to competing binocularly driven neural representations of each rivalrous stimulus. An alternative view posits monocular neural competition together with a switch in eye dominance at the moment of each stimulus swap between eyes. Here, a range of experimental conditions was tested that would change the colors seen if mediated by eye dominance but not if by competition between binocularly driven responses. Observers viewed multiple chromatically rivalrous discs in various temporal and spatial patterns, and reported when all discs in view appeared the same color. Unlike typical ISR paradigms that swap the complete stimulus in each eye, some of the rivalrous discs were swapped at a different time, or faster frequency, than other discs. Monocular dominance of one eye at a time implies that all discs will rarely be seen as identical in color when some discs swap at a different frequency than others. On the other hand, competing binocularly driven neural responses are not affected by asynchronous swap timing among the individual discs. Results for every observer are in accord with competing responses at the level of binocularly driven, chromatically tuned neurons. Although an account based on eye dominance can be constructed using many small retinotopic zones that have independent timing for the moment of switching the dominant eye, competing binocularly driven responses are a more parsimonious explanation.

Decoding chromaticity and luminance from patterns of EEG activity.

A long-standing question in the field of vision research is whether scalp-recorded EEG activity contains sufficient information to identify stimulus chromaticity. Recent multivariate work suggests that it is possible to decode which chromaticity an observer is viewing from the multielectrode pattern of EEG activity. There is debate, however, about whether the claimed effects of stimulus chromaticity on visual evoked potentials (VEPs) are instead caused by unequal stimulus luminances, which are achromatic differences. Here, we tested whether stimulus chromaticity could be decoded when potential confounds with luminance were minimized by (1) equating chromatic stimuli in luminance using heterochromatic flicker photometry for each observer and (2) independently varying the chromaticity and luminance of target stimuli, enabling us to test whether the pattern for a given chromaticity generalized across wide variations in luminance. We also tested whether luminance variations can be decoded from the topography of voltage across the scalp. In Experiment 1, we presented two chromaticities (appearing red and green) at three luminance levels during separate trials. In Experiment 2, we presented four chromaticities (appearing red, orange, yellow, and green) at two luminance levels. Using a pattern classifier and the multielectrode pattern of EEG activity, we were able to accurately decode the chromaticity and luminance level of each stimulus. Furthermore, we were able to decode stimulus chromaticity when we trained the classifier on chromaticities presented at one luminance level and tested at a different luminance level. Thus, EEG topography contains robust information regarding stimulus chromaticity, despite large variations in stimulus luminance.

Does feature integration affect resolution of multiple simultaneous forms of ambiguity?

Ambiguity resolution, perceptual grouping, and feature integration all occur seamlessly and subconsciously. When multiple regions of an image share ambiguous features, perceptual grouping can yield an integrated object percept rather than one of multiple objects, each with its individual features. Here, perceptual resolution and grouping of chromatically rivalrous Necker cubes were investigated in three experiments to determine the principles that underlie these coherent percepts. The first experiment showed perceptual grouping beyond independent resolution of each cube's color and orientation, but the second experiment did not show grouping greater than expected from separate color- and orientation-grouping processes. The third experiment found no reliable difference in grouping when two features (color and orientation) were part of the same object versus when they were distributed across separate objects. These findings fail to support a role for feature conjunctions in grouping objects with multiple ambiguous features.

Grouping ambiguous neural representations: neither identical chromaticity (the stimulus) nor color (the percept) is necessary.

Multiple regions, each with the same ambiguous chromatic neural representation, are resolved to have the identical perceived color more often than chance [Proc. Natl. Acad. Sci. USA93, 15508 (1996)PNASA60027-842410.1073/pnas.93.26.15508; J. Opt. Soc. Am. A35, B85 (2018)JOAOD60740-323210.1364/JOSAA.35.000B85]. This reveals that the regions are grouped, but it is unclear whether they are grouped because each one has the identical competing representations of the same stimuli (that is, the same chromaticities) or, alternatively, identical competing representations of the same colors one sees. The current study uses chromatic induction, as in Nat. Neurosci.6, 801 (2003)NANEFN1097-625610.1038/nn1099, to disentangle whether grouping depends on identical (though ambiguous) stimulus chromaticities or on perceived colors, by (1) inducing one chromaticity to appear in two different colors or (2) inducing two different chromaticities to appear in the same color. All stimuli were equiluminant gratings with chromatic inducing and test fields. Three observers were tested, first completing color matches to measure induced color-appearance shifts and second completing grouping measurements using interocular-switch rivalry, a method with rivalrous dichoptic images swapped between the eyes at 3.75 Hz [J. Vis.17, 9 (2017)1534-736210.1167/17.5.9]. Each of two separate areas, one above and one below fixation, had dichoptic rivalry. The two sets of regions had either identical or different chromaticities that could appear either as the same color or not. Observers reported their percepts when both areas above and below fixation were grouped by color or by chromaticity (or neither in an additional experimental condition). All conditions showed significant groupings for every observer, including when neither color nor chromaticity was identical in a "group." Moreover, there was never a significant effect of chromaticity versus color for any observer. This is the result expected if neither color nor chromaticity must match between two regions in order for them to be grouped and suggests that, instead, some other feature drives grouping.

Seeing fruit on trees: enhanced perceptual dissimilarity from multiple ambiguous neural representations.

Perceptual grouping contributes to the resolution of visual ambiguity of multiple spatially separate regions in view by enhancing their perceptual similarity. Here, the same ambiguous neural representations are shown also to enhance perceived dissimilarity among the regions. Two separated equiluminant gratings were made ambiguous by introducing rivalry for one of two of their features: orientation or chromaticity. Observers perceived two gratings (above and below fixation) to be different in both color and orientation more often than chance. Overall, a disambiguating process was found to select often for maximal perceived dissimilarity between two objects.

Neural representations of perceptual color experience in the human ventral visual pathway.

Color is a perceptual construct that arises from neural processing in hierarchically organized cortical visual areas. Previous research, however, often failed to distinguish between neural responses driven by stimulus chromaticity versus perceptual color experience. An unsolved question is whether the neural responses at each stage of cortical processing represent a physical stimulus or a color we see. The present study dissociated the perceptual domain of color experience from the physical domain of chromatic stimulation at each stage of cortical processing by using a switch rivalry paradigm that caused the color percept to vary over time without changing the retinal stimulation. Using functional MRI (fMRI) and a model-based encoding approach, we found that neural representations in higher visual areas, such as V4 and VO1, corresponded to the perceived color, whereas responses in early visual areas V1 and V2 were modulated by the chromatic light stimulus rather than color perception. Our findings support a transition in the ascending human ventral visual pathway, from a representation of the chromatic stimulus at the retina in early visual areas to responses that correspond to perceptually experienced colors in higher visual areas.

Perceptual resolution of ambiguous neural representations for form and chromaticity.

A coherent percept of our visual world is important for functioning. Ambiguities, however, are implicit in visual neural representations and must be resolved for stable perception of objects and scenes. Grouping processes can link multiple neurally ambiguous fragments across the visual field. Experiments here determined how multiple visual features of each fragment contribute to perceptual resolution of ambiguity by grouping. Chromatic interocular-switch rivalry, a technique for presenting competing dichoptic images, was used to induce ambiguous neural representations for equiluminant chromatic discs and gratings. Two dichoptic stimuli were presented simultaneously to measure the amount of time they both appeared the same in at least one feature domain. The two stimuli were grouped when they appeared to share ambiguous features such as color, orientation, and spatial frequency more often than chance. Experiments here tested whether unshared and unambiguous features impeded grouping of the ambiguous components. Overall, the results show that grouping can be driven by neural ambiguity that is common for fragments across the visual field, even when the fragments also have other unshared, unambiguous features.

Ambiguous chromatic neural representations: Perceptual resolution by grouping.

Two basic principles of human color vision are (1) color is not in light but instead constructed within the (human) perceiver and (2) in natural viewing, photoreceptor signals fail to determine uniquely the colors we see. The visual system, therefore, must resolve the ambiguity implicit in the neural representation of the stimulus. This paper focuses on a perceptual property used by the visual system to resolve ambiguity when two or more parts of a scene share the same ambiguous chromatic neural representation: the parts are grouped so tend to appear the same color. Chromatic interocular-switch rivalry, a novel paradigm, is described and then used to demonstrate grouping of two, four or 16 parts of an image with the same ambiguous chromatic representation.

Perceptual resolution of color for multiple chromatically ambiguous objects.

In a classic study, Kovács et al. [Proc. Natl. Acad. Sci. USA93, 15508 (1996)PNASA60027-842410.1073/pnas.93.26.15508] used an array of many disks presented dichoptically with half of the disks in one eye "red" and the other half "green;" disk chromaticities in the fellow eye were reversed, resulting in binocular color rivalry for every disk, thus creating color ambiguity. Surprisingly, the binocularly fused percept sometimes was all disks of the same color (red or green), which showed that perceptual resolution of the many ambiguous neural representations did not rely completely on monocular dominance or on independent resolution for each disk. The present study replicates and expands on the original with the aim to isolate binocularly driven neural mechanisms of perceptual resolution without contamination from monocular dominance. Observers viewed a color-rivalrous array with 16 disks presented either steadily to each eye, as in Kovács et al., or with chromatic interocular-switch rivalry (CISR), which swaps the two images between the eyes every 133 ms. The total proportion of viewing time when the 16 disks were perceived to be all red or all green was measured. For three observers, the disks all appeared the same color more often with CISR than with steady rivalrous presentation, suggesting that monocular dominance interferes with grouped perceptual resolution of ambiguous stimuli in the Kovács paradigm. This conclusion was supported by an additional condition using CISR, but with every disk the same color in one eye at each instant (e.g., all "red" disks in one eye and all "green" in the other). This condition was never significantly different from the original CISR condition, as expected if CISR reveals only binocularly mediated perceptual resolution of the disks' color, irrespective of monocular neural representations. In conclusion, chromatically tuned binocularly driven neurons account for perceptual resolution of CISR.

Chromatic induction in space and time.

The color appearance of a light depends on variation in the complete visual field over both space and time. In the spatial domain, a chromatic stimulus within a patterned chromatic surround can appear a different hue than the same stimulus within a uniform surround. In the temporal domain, a stimulus presented as an element of a continuously changing chromaticity can appear a different color compared to the identical stimulus, presented simultaneously but viewed alone. This is the flash-lag effect for color, which has an analog in the domain of motion: a pulsed object seen alone can appear to lag behind an identical pulsed object that is an element of a motion sequence. Studies of the flash-lag effect for motion have considered whether it is mediated by a neural representation for the moving physical stimulus or, alternatively, for the perceived motion. The current study addresses this question for the flash-lag effect for color by testing whether the color flash lag depends on a representation of only the changing chromatic stimulus or, alternatively, its color percept, which can be altered by chromatic induction. METHODS: baseline measurements for spatial chromatic induction determined the chromaticity of a flashed ring within a uniform surround that matched a flashed ring within a patterned surround. Baseline measurements for the color flash-lag effect determined the chromaticity of a pulsed ring presented alone (within a uniform surround) that matched a pulsed ring presented in a sequence of changing chromaticity over time (also within a uniform surround). Finally, the main experiments combined chromatic induction from a patterned surround and the flash-lag effect, in three conditions: (1) both the changing and pulsed rings were within a patterned chromatic surround; (2) the changing ring was within a patterned surround and the pulsed ring within a uniform surround; and (3) the changing ring was within a uniform surround and the pulsed ring within a patterned surround. RESULTS: the flash-lag measurements for a changing chromaticity were affected by perceptual changes induced by the surrounding chromatic pattern. Thus, the color shifts induced by a chromatic surround are incorporated in the neural representation mediating the flash-lag effect for color.

Illusory edges comingle with real edges in the neural representation of objects.

The visual system must transform a point-by-point biological representation from the photoreceptors into neural representations of separate objects. Even a uniform circular patch of light that slowly modulates in luminance can be segmented into separate central and surrounding areas merely by introducing black lines to outline a central square. The black lines cause brightness induction in the center even though the light inside and outside the square is always identical, as predicted by spatial antagonism between the square central area and its surround. Importantly, illusory Kanizsa lines forming the square are as effective for this brightness induction as real black lines, suggesting a 'form-cue invariant' cortical neural representation that does not distinguish between a central region set off by real or illusory edges. An open question is whether separate subsystems generate objects defined by real versus illusory edges, each providing the same form-cue invariant neural representation of an object, or whether form-cue invariance extends to integrating component pieces that together define an object. Experiments here show object segmentation when subparts of a square are defined by a mixture of real and illusory edges. Subjects adjusted the Michelson contrast of a separate patch to match the perceived modulation depth within the central region of a circular field that slowly oscillated in luminance. A closed, four-sided figure, no matter how constructed, reduced the perceived modulation depth within the central region. This shows that both real and illusory subparts can be integrated to segment center from surround. It supports a strong version of form-cue invariance in which neural mechanisms responsible for object segmentation are impartial to the piecemeal cues that are integrated to define an object.

Color opponency: tutorial.

In dialogue, two color scientists introduce the topic of color opponency, as seen from the viewpoints of color appearance (psychophysics) and measurement of nerve cell responses (physiology). Points of difference as well as points of convergence between these viewpoints are explained. Key experiments from the psychophysical and physiological literature are covered in detail to help readers from these two broad fields understand each other's work.

Chromatic interocular-switch rivalry.

Interocular-switch rivalry (also known as stimulus rivalry) is a kind of binocular rivalry in which two rivalrous images are swapped between the eyes several times a second. The result is stable periods of one image and then the other, with stable intervals that span many eye swaps (Logothetis, Leopold, & Sheinberg, 1996). Previous work used this close kin of binocular rivalry with rivalrous forms. Experiments here test whether chromatic interocular-switch rivalry, in which the swapped stimuli differ in only chromaticity, results in slow alternation between two colors. Swapping equiluminant rivalrous chromaticities at 3.75 Hz resulted in slow perceptual color alternation, with one or the other color often continuously visible for two seconds or longer (during which there were 15+ eye swaps). A well-known theory for sustained percepts from interocular-switch rivalry with form is inhibitory competition between binocular neurons driven by monocular neurons with matched orientation tuning in each eye; such binocular neurons would produce a stable response when a given orientation is swapped between the eyes. A similar model can account for the percepts here from chromatic interocular-switch rivalry and is underpinned by the neurophysiological finding that color-preferring binocular neurons are driven by monocular neurons from each eye with well-matched chromatic selectivity (Peirce, Solomon, Forte, & Lennie, 2008). In contrast to chromatic interocular-switch rivalry, luminance interocular-switch rivalry with swapped stimuli that differ in only luminance did not result in slowly alternating percepts of different brightnesses.

Chromatic Information and Feature Detection in Fast Visual Analysis.

The visual system is able to recognize a scene based on a sketch made of very simple features. This ability is likely crucial for survival, when fast image recognition is necessary, and it is believed that a primal sketch is extracted very early in the visual processing. Such highly simplified representations can be sufficient for accurate object discrimination, but an open question is the role played by color in this process. Rich color information is available in natural scenes, yet artist's sketches are usually monochromatic; and, black-and-white movies provide compelling representations of real world scenes. Also, the contrast sensitivity of color is low at fine spatial scales. We approach the question from the perspective of optimal information processing by a system endowed with limited computational resources. We show that when such limitations are taken into account, the intrinsic statistical properties of natural scenes imply that the most effective strategy is to ignore fine-scale color features and devote most of the bandwidth to gray-scale information. We find confirmation of these information-based predictions from psychophysics measurements of fast-viewing discrimination of natural scenes. We conclude that the lack of colored features in our visual representation, and our overall low sensitivity to high-frequency color components, are a consequence of an adaptation process, optimizing the size and power consumption of our brain for the visual world we live in.

Color-motion feature-binding errors are mediated by a higher-order chromatic representation.

Peripheral and central moving objects of the same color may be perceived to move in the same direction even though peripheral objects have a different true direction of motion [Nature429, 262 (2004)10.1038/429262a]. The perceived, illusory direction of peripheral motion is a color-motion feature-binding error. Recent work shows that such binding errors occur even without an exact color match between central and peripheral objects, and, moreover, the frequency of the binding errors in the periphery declines as the chromatic difference increases between the central and peripheral objects [J. Opt. Soc. Am. A31, A60 (2014)JOAOD60740-323210.1364/JOSAA.31.000A60]. This change in the frequency of binding errors with the chromatic difference raises the general question of the chromatic representation from which the difference is determined. Here, basic properties of the chromatic representation are tested to discover whether it depends on independent chromatic differences on the l and the s cardinal axes or, alternatively, on a more specific higher-order chromatic representation. Experimental tests compared the rate of feature-binding errors when the central and peripheral colors had the identical s chromaticity (so zero difference in s) and a fixed magnitude of l difference, while varying the identical s level in center and periphery (thus always keeping the s difference at zero). A chromatic representation based on independent l and s differences would result in the same frequency of color-motion binding errors at everyslevel. The results are contrary to this prediction, thus showing that the chromatic representation at the level of color-motion feature binding depends on a higher-order chromatic mechanism.

Monocular and binocular mechanisms mediating flicker adaptation.

Flicker adaptation reduces subsequent temporal contrast sensitivity. Recent studies show that this adaptation likely results from neural changes in the magnocellular visual pathway, but whether this adaptation occurs at a monocular or a binocular level, or both, is unclear. Here, two experiments address this question. The first experiment exploits the observation that flicker adaptation is stronger at higher than lower temporal frequencies. Observers' two eyes adapted to 3Hz flicker with an incremental pulse at 1/4 duty cycle, either in-phase or out-of-phase in the two eyes. At the binocular level, the flicker rate was 6Hz in the out-of-phase condition if the two eyes' pulse trains sum. Similar sensitivity reduction was found in both phase conditions, as expected for independent monocular adapting mechanisms. The second experiment tested for interocular transfer of adaptation between eyes. Results showed that (1) flicker adaptation was strongest with adapting and test fields in only the same eye, (2) adaptation can be partially transferred interocularly with adaptation in only the opposite eye, and (3) adaptation was weakened when both eyes were adapted simultaneously at different contrasts, compared to test-eye adaptation alone. Taken together, the findings are consistent with mechanisms of flicker adaptation at both the monocular and binocular level.

The role of color in motion feature-binding errors.

Color-motion feature-binding errors occur in the periphery when half of the objects are red and move downward, and the other half are green and move upward. When red and green objects in the central visual field are similar but move in the opposite directions (red upward, green downward), peripheral objects often take on the perceived motion direction of the like-colored central objects (Wu, Kanai, & Shimojo, 2004). The present study determined whether color is essential to elicit these motion-binding errors, and tested two hypotheses that attempt to explain them. One hypothesis holds that binding errors occur because peripheral and central objects become linked if they have combinations of features in common. A peripheral object's link to central objects overwhelms its posited weak peripheral representation for motion feature binding, so the peripheral object appears to move in the direction of the linked central objects. Eliminating color by making all stimuli achromatic, therefore, should not increase peripheral binding errors. An alternative hypothesis is that binding errors depend on the overall feature correspondence among central and peripheral features represented at a preconjunctive level. In this case, binding errors may increase when all objects are changed to achromatic because chromatic central/peripheral correspondence is maximal (100%). Experiments showed more motion-binding errors with all-achromatic objects than with half red and half green objects. This and additional findings imply that peripheral motion-binding errors (a) can be elicited without color and (b) depend at least in part on the similarity of central and peripheral features represented preconjunctively.